Trilinear Tensor: The Fundamental Construct of Multiple-view Geometry and Its Applications

The topic of representation, recovery and manipulation of three-dimensional (3D) scenes from two-dimensional (2D) images thereof, provides a fertile ground for both intellectual theoretically inclined questions related to the algebra and geometry of the problem and to practical applications such as Visual Recognition, Animation and View Synthesis, recovery of scene structure and camera ego-motion, object detection and tracking, multi-sensor alignment, etc. The basic materials have been known since the turn of the century, but the full scope of the problem has been under intensive study since 1992, rst on the algebra of two views and then on the algebra of multiple views leading to a relatively mature understanding of what is known as \multilinear matching constraints", and the \trilinear tensor" of three or more views. The purpose of this paper is, rst and foremost, to provide a coherent framework for expressing the ideas behind the analysis of multiple views. Secondly, to integrate the various incremental results that have appeared on the subject into one coherent manuscript.